Repository navigation
fix(effects): paginate FHIR and REST read-backs and verify total count (#1476) - #1511
Closed
yunusarda-sudo wants to merge 2 commits into
Closed
yunusarda-sudo wants to merge 2 commits into
yunusarda-sudo wants to merge 2 commits into
Conversation
- Follow FHIR Bundle.link[rel=next] across pages up to max_pages cap - Support REST pagination via RFC 5988 Link header and body next fields - Verify Bundle.total and REST total/total_count against records read, returning INDETERMINATE on mismatch - Add pagination and total validation test coverage for FHIR and REST verifiers - Resolves OpenAdaptAI#1476 (realism audit rank 11)
Signed-off-by: Yunus Ardaguney <yunusarda-sudo@users.noreply.github.com>
Member
|
Thank you for this, it's a real gap from our audit (#1476), and your fail-closed handling of failed pages, the page cap and mismatched totals is right. We're closing it as superseded by #1520, which rewrites the same two functions (
We'll credit you in #1520 and the changelog. Your #1515 is approved and queued. Thanks again, and contributions like these are very welcome. |
This was referenced Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Bug fix
Summary
Fixes #1476 (
docs/audits/REALISM_AUDIT.mdRank 11).In
openadapt_flow/runtime/effects/fhir.pyandrest.py, effect verifiers previously inspected only the initial page returned by the system-of-record search query.When a FHIR server or REST endpoint paginates search results (e.g. at 20–50 items), records situated on page 2 and beyond were completely invisible to the read-back. This led to safety flaws:
expected_count=1.Bundle.totalor JSONtotal/total_count, discrepancies between read records and reported totals were ignored.Changes
FHIR Pagination (
openadapt_flow/runtime/effects/fhir.py):FhirEffectVerifier._search(), parseBundle.linkfor relation"next".nextURLs up to a configurable safety limit (max_pages=50, default).expected_total(fromBundle.total) against the count of flattened resources read.totaldiffers fromlen(out)or if pagination stops prematurely, returnNonesojudge_recordsreturnsINDETERMINATE.REST Pagination (
openadapt_flow/runtime/effects/rest.py):RestRecordVerifier._fetch_records(), inspect RFC 5988 / 8288 standardLink: <url>; rel="next"header (viaresp.links["next"]["url"]) and JSON body attributes (next,next_url,links.next).max_pages=50.expected_total(body["total"]\orbody["total_count"]) against accumulated records, returningNone(INDETERMINATE) on mismatch.Test Coverage:
tests/_fhir_fake.pywith_count,_offset, and_bad_totalquery parameters.tests/test_effect_fhir.pyverifying multi-page accumulation, absence detection past page 1, and total mismatch rejection.tests/test_rest_verifier_transport.pycovering header Link pagination, body JSON link pagination, and total mismatch handling.Checklist
pytest packages/flow/tests/test_effect_fhir.py packages/flow/tests/test_rest_verifier_transport.py-> 17 passed, 1 skipped)ruff checkclean)Notes
max_pagesdefaults to 50 as a protective bounded loop against infinite server cycles.totaldoes not match the retrieved count, read-back treats the evidence as indeterminate rather than making false claims.